Methods for performing biometric recognition of a human eye and corroboration of same

ABSTRACT

A method of biometric recognition is provided. Multiple images of the face or other non-iris image and iris of an individual are acquired. If the multiple images are determined to form an expected sequence of images, the face and iris images are associated together. A single camera preferably acquires both the iris and face images by changing at least one of the zoom, position, or dynamic range of the camera. The dynamic range can be adjusted by at least one of adjusting the gain settings of the camera, adjusting the exposure time, and/or adjusting the illuminator brightness. The expected sequence determination can be made by determining if the accumulated motion vectors of the multiple images is consistent with an expected set of motion vectors and/or ensuring that the iris remains in the field of view of all of the multiple images.

RELATED APPLICATIONS

This application is a continuation of, and claims priority to U.S.application Ser. No. 14/336,724 filed Jul. 21, 2014, entitled “Methodsfor Performing Biometric Recognition of a Human Eye and Corroboration ofthe Same”, which is a continuation of, and claims priority to:

U.S. application Ser. No. 13/800,496, filed Mar. 13, 2013, entitled“Methods for Performing Biometric Recognition of a Human Eye andCorroboration of the Same”, issued as U.S. Pat. No. 8,818,053 on Aug.26, 2014 which is a continuation of, and claims priority to:

U.S. Ser. No. 13/567,901, filed Aug. 6, 2012, entitled “Methods forPerforming Biometric Recognition of a Human Eye and Corroboration of theSame”, issued as U.S. Pat. No. 8,798,330 on Aug. 5, 2014 which is acontinuation of, and claims the benefits of and priority to:

U.S. application Ser. No. 12/887,106 filed Sep. 21, 2010 entitled“Methods for Performing Biometric Recognition of a Human Eye andCorroboration of the Same”, issued as U.S. Pat. No. 8,260,008 on Sep. 4,2012 which is a continuation-in-part of, and claims priority to:

U.S. application Ser. No. 11/559,381 filed Nov. 13, 2006, entitled“Apparatus and Methods for Detecting the Presence of a Human Eye”,issued as U.S. Pat. No. 7,801,335 on Sep. 21, 2010 which claims priorityto:

U.S. Provisional Application No. 60/597,130 filed on Nov. 11, 2005,entitled “Measuring the Geometrical Relationship between ParticularSurfaces of Objects”, and U.S. Provisional Application No. 60/597,152filed on Nov. 14, 2005, entitled “Measuring the Geometrical andPhotometric Relationships between or within Particular Surfaces ofObjects”, and U.S. Application No. 60/597,231, filed on Nov. 17, 2005entitled “Method for associating Face and Iris Imagery”, and U.S.Provisional Application No. 60/597,289, filed on Nov. 21, 2005, entitled“Method for Reliable Iris Matching Using a Personal Computer andWeb-Camera”, and U.S. Provisional Application No. 60/597,336 filed onNov. 25, 2005, entitled “Methodology for Detecting Non-Live Irises in anIris Recognition System” each of them is incorporated herein byreference in their entirety.

BACKGROUND OF THE DISCLOSURE

Field of the Disclosure

This disclosure relates generally to systems in which imagery isacquired primarily to determine or verify the identity of a person usinga biometric recognition system, and more specifically to systems inwhich there is a need to detect the presence of a live, human eye in theimagery. The biometric used for recognition may be the iris, forexample.

Description of Related Art

Like a fingerprint, an iris can be used to uniquely identify a person. Anumber of systems have been implemented for this purpose. For oneexample, U.S. Pat. No. 4,641,349, titled “Iris Recognition System,”issued to Flom et al. on Feb. 3, 1987, and U.S. Pat. No. 5,291,560,titled “Biometric Personal Identification Based on Iris Analysis,”issued to Daugman on Mar. 1, 1994, discloses a system for identifying aperson based upon unique characteristics of the iris. A camera capturesan image of the iris, the iris is segmented, and then the iris portionis normalized to compensate for pupil dilation. The normalized irisfeatures are then compared with previously stored image information todetermine whether the iris matches.

For another example, U.S. Pat. No. 5,572,596, titled “Automated,Non-Invasive Iris Recognition System and Method,” issued to Wildes etal. on Nov. 5, 1996, discloses an alternate method of performing irisrecognition using normalized correlation as a match measure. Furtheradvantages and methods are set forth in detail in this patent.

For another example, U.S. Pat. No. 6,247,813, titled “IrisIdentification System and Method of Identifying a Person through IrisRecognition,” issued to Kim et al. on Jun. 19, 2001, discloses anothersystem used for iris recognition, which implements a uniqueidentification methods. The system divides a captured image of an irisinto segments and applies a frequency transformation. Further details ofthis method are set forth in the patent.

For yet another example, U.S. Pat. No. 6,714,665, titled “FullyAutomated Iris Recognition Systems Utilizing Wide and Narrow Fields ofView,” issued to Hanna et al. on Mar. 30, 2004, discloses a systemdesigned to automatically capture and identify a person's iris. Thissystem uses a camera with a wide field of view to identify a person anda candidate iris. Once identified, a second camera with a narrow fieldof view is focused on the iris and an image captured for irisrecognition. Further details of this method are set forth in the patent.

One problem faced by iris recognition systems involves the possibilityof spoofing. Specifically, a life-sized, high-resolution photograph of aperson may be presented to an iris recognition system. The irisrecognition systems may capture an image of this photograph and generatea positive identification. This type of spoofing presents an obvioussecurity concerns for the implementation of an iris recognition system.One method of addressing this problem has been to shine a light onto theeye, then increase or decrease the intensity of the light. A live, humaneye will respond by dilating the pupil. This dilation is used todetermine whether the iris presented for recognition is a live, humaneye or merely a photograph—since the size of a pupil on a photographobviously will not change in response to changes in the intensity oflight. One disadvantage of this type of system involves the timerequired to obtain and process data as well as the irritation a personmay feel in response to having a light of varying intensity shone intotheir eye.

U.S. Pat. No. 6,760,467, titled “Falsification Discrimination Method forIris Recognition System,” issued to Min et al. on Jul. 6, 2004, attemptsto address this problem. This system positions a pair of LED's onopposite sides of a camera. These LED's are individually lighted andimages captured through a camera. These images are analyzed to determinewhether light from the LED's was reflected back in a manner consistentwith a human eye. Because a flat photograph will not reflect light backin the same manner, this system aims to deter this type of spoofing. Onedisadvantage of this system, involves the simplicity of the approach andthe placement of the LED's. With two LED's positioned at a fixed, knownlocation, the method can be defeated by appropriate placement of twosmall illuminators in an iris image. Also, while this system may operatemore quickly than systems that dilate a pupil, it still requires time tocapture at least two separate images: one when each of the two LED's areindividually lit. Further, a third image needs to be captured if thesystem requires both LED's to be illuminated to capture imagery that issufficiently illuminated for recognition.

The above identified patents are each incorporated herein by referencein their entirety as well as each of the patents and publicationsidentified below.

As mentioned above, it is well known that imagery of the iris can bereliably matched to previously recorded iris imagery in order to performreliable verification or recognition. For example, see Daugman J (2003)“The importance of being random: Statistical principles of irisrecognition.” Pattern Recognition, vol. 36, no. 2, pp 279-291. Howeversince the iris patterns are not easily recognizable to a human, it isimpossible to demonstrate to a user who has been rejected from any irisrecognition system the reason for the rejection. On the other hand, if aface image of the person whose iris has been used for recognition isacquired, it is easy to demonstrate the reason for rejection since faceimagery can be easily interpreted by humans. Therefore, especially inunattended systems, there is a need for a highly secure method ofassociating an acquired face image to an acquired iris image, preferably(although not necessarily) with just one sensor in order to reduce costand size of the solution.

SUMMARY

This summary is provided solely to introduce a more detailed descriptionof the invention as shown in the drawings and explained below.

Apparatus and methods for detecting a human iris use a computer screenon which an image is presented. The image is reflected off of a person'seye. The reflection is analyzed to determine whether changes to thereflected image are consistent with a human eye.

According to one aspect of the invention, a human eye is detected bypresenting a first image on a computer screen that is oriented to face auser. At least one camera (and in some preferred embodiments at leasttwo cameras) is positioned near the computer screen and oriented to facethe user so that light emitted by the computer screen as the first imageis reflected by the user and captured by the camera as a second image.The camera may be attached as part of the computer screen or separatelymounted. A computer is operably coupled with the computer screen and thecamera and the computer detects a human eye when at least a portion ofthe second image includes a representation of the first image on thecomputer screen reflected by a curved surface consistent with a humaneye. The computer may be operated locally or operated remotely andconnected through a network.

According to further aspects of the invention, the human eye is detectedwhen the representation of the first image included in the second imageis approximately equal to a human-eye magnification level, which isdetermined by dividing 3 to 6 millimeters by a distance from thecomputer screen to the user. For an implementation where the user is atleast 100 millimeters from the computer screen, the representation ofthe first image is at least ten times smaller than the first image. Foran implementation where the user is approximately 75 to 500 millimetersfrom the computer screen and the camera, the representation of the firstimage is approximately 12.5 to 166.7 times smaller than the first image.The determination can further require that the magnification at thecenter of the representation is smaller than a magnification in areassurrounding the center of the representation. Likewise, thedetermination can detect a human eye when the second image includes therepresentation of the first image on the computer screen reflected by anellipsoidal surface with an eccentricity of approximately 0.5 and aradius of curvature at the apex of the surface of approximately 7.8millimeters.

According to further aspects of the invention, the portion of the secondimage containing a representation is isolated. The comparison is madebetween the first image and the portion of the second image containingthe human iris. In addition or alternatively, the determination can bemade by searching the second image for a warped version of the firstimage. For example, a checkered pattern may be presented on the computerscreen. The second image is then searched for a warped version of thecheckered pattern.

According to further aspects of the invention, a third image ispresented on the computer screen that is different than the first image.For example, the first image may be a checkered pattern and the thirdimage may also be a checkered pattern but with a different arrangementof checkered squares. A fourth image is captured through the camera(s).The computer then aligns the second and fourth image. The computer thendetermines a difference image representing the difference between thesecond image and the fourth image. The portion of the differencecontaining an eye, and thus containing a reflection of the first and thethird image are isolated. This may be found by identifying the portionof the difference image containing the greatest difference between thesecond and fourth images. A human eye is detected when the portion ofthe difference image is consistent with a reflection formed by a curvedsurface. For example, this can be detected determining the size of theportion containing a reflection of the first and third images; where theratio between the image size and the image reflection size is greaterthan 10 to 1 then a human eye is detected. This ratio can be calculatedfor a particular application by dividing the distance between the userand the computer screen by approximately 3 to 6 millimeters, where thecamera is at or near the computer screen.

According to still further aspects of the invention, a skin area isfound in the second image and a determination is made as to whether thereflection of light from the skin area is consistent with human skin.

According to another aspect of the invention, a human eye is detected bypresenting a first image on a computer screen positioned in front of auser. A first reflection of the first image off of the user is capturedthrough a camera. The computer screen presents a second image on thecomputer screen positioned in front of the user. The camera captures asecond reflection of the second image off of the user. The first andsecond images can be, for example, a checkered pattern of colors wherethe second image has a different or inverted arrangement. A computercompares the first reflection of the first image with the secondreflection of the second image to determine whether the first reflectionand the second reflection were formed by a curved surface consistentwith a human eye. This comparison can be made, for example, by aligningthe first reflection and the second reflection then calculating adifference between them to provide a difference image. The portion ofthe difference image containing a difference between a reflection of thefirst image and a reflection of the second image is identified. The sizeof this portion is determined. A human eye is detected when the ratio ofthe size of this portion to the size of the first and second image isapproximately equal to a human-eye magnification level. Where the camerais located at or near the computer screen, the human-eye magnificationlevel is determined by dividing the distance from the computer screen tothe user by approximately 3 to 6 millimeters.

According to another aspect of the invention, a human eye is detected byobtaining a first image of a user positioned in front of a computerscreen from a first perspective and obtaining a second image of the userpositioned in front of the computer screen from a second perspective. Acomputer identifies a first portion of the first image and a secondportion of the second image containing a representation of a human eye.The computer detects a human eye when the first portion of the firstimage differs from the second portion of the second image. For example,the computer may detect changes in specularity consistent with a humaneye. For another example, the computer may align the first image withthe second image and detect an area of residual misalignment. In thiscase, a human eye is detected if this area of residual misalignmentexceeds a predetermined threshold.

According to further aspects of the invention, the first perspective isobtained by presenting a first graphic on the computer screen at a firstlocation and instructing the user to view the first image. The secondperspective is obtained by presenting a second graphic on the computerscreen at a second location, different than the first, and instructingthe user to view the second image.

According to another aspect of the invention, a human eye is detected bypresenting one or more illuminators oriented to face a user. At leastone camera is positioned proximate the illuminators. The camera(s) isoriented to face the user so that light emitted by the illuminators isreflected by the user and captured by the camera(s). The camera(s) alsoobtain a second image through at a different time than the first image.A computer detects a first position of a reflection in the first imageand a second position of a reflection in the second image. The computernormalizes any positional change of the user in the first image and thesecond image based upon the first position and the second position. Thisnormalizing includes compensating for motion during the time between thefirst image and the second image by using at least a translation motionmodel to detect residual motion of the position of the reflection. Ahuman eye is detected when a change between the first image and thesecond image is consistent with reflection by a curved surfaceconsistent with that of a human eye.

In another aspect of the invention, the invention includes a method ofbiometric recognition that associates face and iris imagery so that itis known that the face and iris images are derived from the same person.The methodology allows face acquisition (or recognition) and irisrecognition to be associated together with high confidence using onlyconsumer-level image acquisition devices.

In general, the inventive method of biometric recognition thatassociates face and iris imagery includes a method of biometricrecognition. Multiple images of the face and iris of an individual areacquired, and it is determined if the multiple images form an expectedsequence of images. If the multiple images are determined to form anexpected sequence, the face and iris images are associated together. Ifthe face and iris images are associated together, at least one of theiris images is compared to a stored iris image in a database.Preferably, the iris image comparison is performed automatically by acomputer. Additionally or in the alternative, if the face and irisimages are associated together, at least one of the face images iscompared to a stored face image in a database. Preferably, the faceimage comparison is performed manually by a human.

Preferably, the acquiring of both face and iris images is performed by asingle sensing device. That single sensing device is preferably a camerathat takes multiple images of a person's face. Optionally, a midpoint ofthe camera's dynamic range is changed while taking the multiple imagesof the person's face. In addition or in the alternative, the position ofthe user relative to the camera is changed while taking the multipleimages of the person's face. In addition or in the alternative, the zoomof the camera is changed while taking the multiple images of theperson's face. Preferably, the acquiring of images occurs at a framerate of at least 0.5 Hz.

To prevent fraudulent usage of the system (e.g., a person inserting aphoto of someone else's iris into the field of view), at least oneimaging parameter is determined from the multiple images acquired, andthe at least one imaging parameter determined from the multiple imagesis compared to at least one predetermined expected imaging parameter. Ifthe at least one imaging parameter determined from the multiple imagesis significantly different from the at least one predetermined expectedimaging parameter, then it is determined that the multiple images do notform an expected sequence. Regarding the at least one imaging parameter,it may include at least one of determining if the accumulated motionvectors of the multiple images is consistent with an expected set ofmotion vectors; or ensuring that the iris remains in the field of viewof all of the multiple images. This preferably takes places atsubstantially the same time as the acquiring step. If it is detectedthat at least one of i) inconsistent accumulated motion vectors or ii)that the iris is not in the field of view of all of the multiple images,then an error message is generated and the acquisition of images ceasesand is optionally reset.

Because the face and the iris have very different reflectivityproperties, the imaging device that captures both face and iris imagesmust be adjusted accordingly. As such, preferably, the sensitivity ofthe camera is altered between a first more sensitive setting foracquiring iris images and a second less sensitive setting for acquiringface images. For example, the altering of the sensitivity may includealternating back and forth between the first and second settings duringthe acquiring step. This alternating step may be performed for everyimage so that every other image is acquired under substantially the samefirst or second setting. Whatever the timing of the altering of thesensitivity of the camera may be, how the altering may be accomplishedmay include at least one of the following: adjusting the gain settingsof the camera; adjusting the exposure time; or adjusting the illuminatorbrightness. Preferably, the first more sensitive setting issubstantially in a range of 1 to 8 times more sensitive than the secondless sensitive setting.

Preferably, the acquiring step of the inventive method is performeduntil at least one face image suitable for human recognition is acquiredand at least one iris image suitable for computer recognition isacquired. The acquisition of the at least one suitable face image ispreferably required to occur within a predetermined amount of time ofthe acquisition of the at least one suitable iris image, either beforeor afterwards.

More generally, the inventive method of biometric recognition includesthe steps of acquiring at least one non-iris image suitable for humanrecognition, and acquiring at least one iris image suitable for computerrecognition within a predetermined period of time from the non-irisimage acquiring step to ensure that both suitable images are from thesame person. The non-iris image includes at least one of a body image, aface image, an identification code image, or a location image.

BRIEF DESCRIPTION OF THE DRAWINGS

Embodiments of the present invention are illustrated by way of example,and not by way of limitation, in the figures and accompanying drawingsand in which like reference numerals refer to similar elements and inwhich:

FIG. 1 is a functional block diagram of one preferred system used todetect a human eye.

FIG. 2 is a flow chart showing one preferred method of detecting a humaneye.

FIG. 3 is a flow chart showing another preferred method of detecting ahuman eye.

FIG. 4 is a sequential schematic diagram illustrating a biometricrecognition method that associates an iris image with a face image inaccordance with an embodiment of the invention.

FIG. 5 is a flow chart illustrating a method for associating face andiris imagery in a sequence in accordance with an embodiment of theinvention.

FIG. 6 is a sequential schematic diagram illustrating a biometricrecognition method that associates an iris image with a face image usingmultiple sensors in accordance with an embodiment of the invention.

DETAILED DESCRIPTION

In the following description, for the purposes of explanation, numerousspecific details are set forth in order to provide a thoroughunderstanding of the embodiments of invention described herein. It willbe apparent, however, that embodiments of the invention may be practicedwithout these specific details. In other instances, well-knownstructures and devices are shown in block diagram form in order to avoidunnecessarily obscuring the description of embodiments of the invention.It should also be noted that these drawings are merely exemplary innature and in no way serve to limit the scope of the invention, which isdefined by the claims appearing hereinbelow.

Functional Overview

In iris recognition applications, the iris is imaged behind thetransparent corneal surface which has a particular convex shape. Lightis also typically reflected off the cornea itself and back into theimager. In addition, the retinal surface is imaged through the pupilalthough it typically appears dark due to the relatively small amount oflight that is returned to the imager from it. In order to determinewhether a detected iris is live, parameters of the geometrical and/orphotometric relationships and/or properties of the iris, retina andcornea determined. The reflective properties of a human eye are detailedat length in “The World in an Eye,” published by Ko Nishino and Shree K.Nayar, in IEEE Conference on Pattern Recognition, Vol 1, pp 444-451,June 2004, which is incorporated by reference in its entirety. In thepresent invention, using these reflective parameters, a determination ismade as to whether an eye is live or not. Following the methodsdisclosed below, various components and their configuration in an irisrecognition system can be varied in order to optimize performance forspecific applications where it is easier to modify some configurationparameters compared to others.

More specifically, preferred techniques are discussed for determiningwhether an image detected through a camera is a live, human eye or afalse representation such as a photograph. One or more images arepresented on a computer screen positioned in front of a user. To deterattempts at spoofing, the image used for this determination may vary.Images presented on the computer screen may include a solid color, aregular or warped checkered pattern, random noise, etc. In addition, anumber of different images may be presented in quick succession so thata person is unable to tell which image is being used for thisdetermination, and is unable to predict which image will be displayed atwhich time. One or more cameras are positioned near the computer. Thecameras are positioned to face a person in front of the computer screen.Due to the relatively sharp curvature of the human eye and particularlythe cornea, the image projected on the computer screen will be reflectedback and captured by the cameras.

The captured image may then be analyzed to determine whether it isconsistent with an image reflected by a human eye. A number of methodsmay be used for this purpose. For example, the image reflected by thecornea and captured by the camera will appear substantially smaller thanthe image presented on the computer screen. A threshold level ofmagnification is set based upon the distance to the person's eye and theaverage radius of curvature for a human eye. If the captured imagecontains a reflection of the image presented on the computer screen andit is consistent in size with the expected size of this reflection, ahuman eye will be detected. This deters spoofing with a photographbecause the flat surface of a photograph will not provide thesubstantial reduction in size caused by a reflection off the surface ofthe cornea. A number of other methods and variations can be used to makethis same determination. These are explained in further detail below.

Configuration

FIG. 1 shows one preferred configuration of components. The cornea A isshown schematically as a curved surface. The iris B is shownschematically as a flat surface behind cornea A. For purposes ofmodeling a human eye, the surface shown as iris B could also representthe retinal surface. The cornea A and iris B are illuminated by one ormore light sources I. In addition, a computer L includes a screen facinga user. The screen projects an image that may change with time. Forexample, the screen may project different wavelengths of light, eachilluminating at different time instants. The illumination from items Iand L are reflected off the cornea and directed into one or more camerasor imagers C. Illumination from the light sources I and the computer Lare transmitted through the cornea, reflected off the iris or retina,re-transmitted through the cornea into the imager C. Since the cornea istransparent, imager C captures an image of the iris or retina and alsohas an image of the illumination I or projected image from computer Lsuperimposed on top of it.

A human eye is identified by capturing image data using a particularconfiguration of a set of components such as those shown in FIG. 1, andto compare the captured image data with data that has been predictedusing knowledge of the expected geometrical or photometric configurationof the components. More specifically, two preferred comparison methodsare described.

In the first method, imagery is captured using at least two differentgeometrical or photometric configurations of components. The capturedimage data (or features derived from the data) acquired using oneconfiguration is then compared to the captured image data or derivedfeatures acquired using the second or further configurations. Thecomputer calculates a set of change parameters that characterize thedifference between the captured image data. The set of change parametersis compared with those change parameters that are predicted usingknowledge of the expected change in geometrical or photometricconfiguration of the components. If the measured change parameters aredifferent from the expected change parameters, then the geometric orphotometric configuration of the corneal and iris or retinal surfacesare not as expected, for example the iris and cornea may appear to lieon the same surface. In this case it can be inferred that the iris isnot live. Similarly, if the corneal surface is not consistent with apartially spherical surface, then again it is known that an iris is notlive.

In another preferred method, imagery is captured using one geometric orphotometric configuration of components. The captured image data (orfeatures derived from the data) is compared with data that is predictedusing absolute knowledge of the expected geometrical or photometricconfiguration of the components. For example, for a given imageprojected on the screen of the computer, a particular illuminationpattern would be expected to appear on the surface of the cornea. Whilethese two methods are described separately, they can be combined.

Method Comparing Two Configurations

Introduction:

With reference to FIG. 1, consider a fixed geometry between a candidateiris B and a candidate cornea A, then changes in the geometricalarrangement and/or photometric properties of the illumination I orprojected image L or position of camera C or position of the iris orcorneal surfaces in coordinate system X, Y, T, where X, Y represent astandard 3D coordinate system and T represents time, results in changein the displacement or characteristics between or within the image ofthe iris B (or retina), and the image reflected off the cornea A. Asshown, the screen of computer L can project a graphical image onto thecurved surface of the cornea A. For example, one or more illuminatorscan illuminate the cornea and iris. Two cameras placed at two slightlydifferent locations then image the iris B and cornea A. Due to thedistance between the iris and the corneal surface, the specularreflection of the illuminator or graphical illumination on the corneaappears in a different location and can be warped or shifted withrespect to the iris image in each camera view due to standard parallaxand also the 3D curvature of the corneal surface, which acts similarlyto a convex mirror lens due to its high curvature and reflectivity. Thisdisplacement or parallax can be measured and used to determine whetheran eye has 3D structure that is consistent with the curvature of thecornea.

This process is further shown in FIG. 2, which can be implemented usingseveral methods. For example, one preferred method is shown in R. Kumar,P. Anandan, and K J. Hanna, “Direct Recovery of Shape from MultipleViews: a Parallax Based Approach,” Proceedings of the 12th IAPRInternational Conference on Pattern Recognition, vol. 1, pp. 685-688,1994, which is incorporated by reference in its entirety. The methodshows how multiple images of a 3D scene can be processed to recover the3D structure of a scene. In the first step, the multiple images arealigned together assuming a 3D planar model. For example, in the case ofiris imagery, the 3D plane may be the iris surface itself. After this 3Dplanar alignment, any residual misalignment is indicative of structurethat does not lie on the 3D plane. In the second step, this residualmisalignment is measured.

In another method of measuring 3D structure, a full model of the sceneis recovered without a 3D planar assumption. This type of method isdisclosed in U.S. Pat. No. 5,259,040, titled “Method for determiningsensor motion and scene structure and image processing system thereof,”which is incorporated herein by reference in its entirety.Notwithstanding that a specularity is not a real structure but an imageartifact, its position in the image changes with viewpoint and thereforeis detected by measuring the residual misalignment. If there issignificant misalignment, as measured by thresholding the residualmisalignment, then there is an indication that a 3D structure ispresent. Methods for thresholding residual misalignments are well-knownand an example is given in “Recovering Motion Fields: An Evaluation ofEight Optical Flow Algorithms” B. Galvin, B. McCane, K. Novins, D.Mason, S. Mills, Proceedings of the British Machine Vision Conference(BMVC), 1998, which is incorporated herein by reference in its entirety.Given the corneal curvature, there is not only a residual misalignment,but the magnitude and distribution of the residual misalignment acrossthe image is consistent with the 3D structure of the cornea. An exampleof modeling the reflected image off a curved surface is given in“Omnidirectional Vision,” by Shree Nayar, British Machine VisionConference, 1998, which is incorporated herein by reference in itsentirety. Another example of modeling the reflected image off the corneais “The World in an Eye,” published by Ko Nishino and Shree K. Nayar, inIEEE Conference on Pattern Recognition, Vol 1, pp 444-451, June 2004,which is incorporated herein by reference in its entirety. In thislatter case a camera observes imagery reflected off the cornea that ismodeled as an ellipsoid. It is shown how the deformation introduced bythe ellipsoid can be removed in order to provide a standardizedperspective image. This standardized perspective image can then beprocessed using standard 3D structure recovery algorithms, as describedearlier in this specification. Parameters for the shape of the corneaare well known. For example, the Gullstrand-LeGrand Eye model notes thatthe radius of the cornea is approximately 6.5 mm-7.8 mm. In anotherexample, in “Adler's Physiology of the Eye: Clinical Application,”Kaufman and Alm editors, published by Mosby, 2003, the radius ofcurvature at the apex of the cornea is noted to be approximately 7.8 mmand the eccentricity of the ellipsoid is approximately 0.5. The samemodel that removes the deformation introduced by the corneal surface canbe used in reverse in order to introduce the expected deformation into astandard geometrical pattern (such as a checkerboard) that can bepresented onto the screen. When this deformed image is reflected off thecornea, it is substantially non-deformed so that the image acquired bythe camera is simply the standard geometrical pattern. This simplifiesthe image processing methods that are required for detecting thepatterns in the acquired imagery.

In another example implementation, the illumination screen or device canbe located close to one of the cameras. The reflection off the retinalsurface appears brighter in the camera located closer to the imager dueto the semi-mirrored surface of the retina, and this also indicateswhether an eye has the appropriate geometric and photometric properties.This approach takes advantage of the “red-eye-effect” whereby a lightsource is reflected off the retina and directly into the camera lens. Ifa second light source is placed at a more obtuse angle to the eye andcamera, then less light will be reflected off the retina, although asimilar quantity of light will be reflected off the face and othersurfaces of the scene that scatter light in all directions (such asurface is Lambertian). Lambertian reflectance is described in Horn,“Robot Vision,” MIT Press, pp. 214-315, which is incorporated herein byreference in its entirety.

Further methods that exploit configuration changes are described below.The methods are separated into two steps: (1) illumination control andimage acquisition; and (2) measuring deformation or change incharacteristics. Further examples of these two steps are now described.

Illumination Control and Image Acquisition:

In steps P and Q in FIG. 2, images are acquired using a particularconfiguration of the candidate cornea A and iris B (or retina), theillumination from light source I and/or the screen on computer L, one ormore imagers C and/or orientation or position X, Y of the candidatecorneal and iris or retinal surfaces at a time T. In steps R and S,images are acquired using a different configuration of the same elementsalso at a time T. An example of how a different orientation X, Y isobtained is by imaging the candidate person at two or more locations asthe person walks or moves through a space. Steps (P, Q) and (R, S) mayoccur sequentially (T is different in this case) or simultaneously (T isequal in this case). An example of a sequential method is to use asingle camera C but to modify the geometric or photometric arrangementof the light source I or image projected on the screen of computer L. Byprojecting an image having a geometrical pattern, the overall securityof the system is dramatically improved because the geometrical patterncan be varied randomly under computer control, and because geometricalpattern reflected by an eye is more difficult to spoof than a simplepoint source of light.

Another example of the sequential method is to create a projected imagethat varies over time—a video sequence for example. The video sequencemay comprise a checkerboard pattern. For example, an example projectionmay have 4×4 black or white squares shown on the screen in a randombinary arrangement. The squares may be pre-deformed as described aboveso that the reflected image off the cornea is close to a perfectcheckerboard pattern.

Another example of a sequential method takes advantage of any combinedmotion of the cornea A and iris or retinal surfaces B. As the candidatecornea and iris or retina move through 3D space over time, differentimages are acquired at different time periods and due to the self-motionof the surfaces, the geometry between the said components changes. Anexample of a simultaneous method is to keep the image or light sourcefixed, but to have two cameras that acquire images from slightlydifferent locations.

Measuring Deformation or Change in Characteristics:

The images captured from steps Q and S in FIG. 2 are then sent to amodule that measures the deformation or changes in characteristicsbetween the image content due to reflection off the corneal surface andimage content due to reflection off the iris or retina. There are manydifferent methods for performing this step. In one preferredimplementation, image alignment is performed using a hierarchical,iterative method such as described by Bergen et al., “HierarchicalModel-Based Motion-Estimation,” European Conference on Computer Vision,1993, which is incorporated herein by reference in its entirety. Forexample, a translation model can be applied between the warped imagesW(Q) and the original images O(R). There are, of course, many othermethods for performing alignment. A difference between the alignedimages is then calculated. The aligned images are then filtered toenhance fine-frequency edges due to the checkerboard patterns and toreduce the magnitude of low frequency edges that may occur due toillumination changes elsewhere in the image. There are also many ways offiltering images in this way. One preferred example is described in “AComputational Approach to Edge Detection,” by John Canny, IEEETransactions on Pattern Analysis and Machine Intelligence, Volume 8,Issue 6, 1986, which is incorporated herein by reference in itsentirety. An image difference is then calculated between the alignedfiltered images to further remove illumination changes and to highlightthe difference between the first reflected filtered image and the secondreflected filtered image. Next, template matching is performed toidentify the unique pattern created by taking the difference of thefiltered, reflected images. If such a pattern is located in the image ofthe correct size and correct deformation, then a live reflection off thecornea has been detected. There are many methods for detecting patternsin images. One preferred example is disclosed in U.S. Pat. No.5,488,675, titled “Stabilizing Estimate of Location of Target RegionInferred from Tracked Multiple Landmark Regions of a Video Image,” andalso U.S. Pat. No. 5,581,629, titled “Method for Estimating the Locationof an Image Target Region from Tracked Multiple Image Landmark Regions,”both of which are incorporated herein by reference in their entirety.

The size of the pattern can also be predicted from the expectedgeometrical shape of the cornea. The detected size of the reflection canthen be measured and used to determine whether the size is consistentwith that of a human cornea. For example, it is known that the focallength of a convex mirror reflector is half the radius of curvature.Using standard lens equations, then 1/f=1/d0+1/d1, where f is the focallength, d0 is the distance of the screen from the cornea, and d1 is thedistance of the reflected virtual image from the cornea. It is knownfrom, for example, “Adler's Physiology of the Eye: ClinicalApplication,” Kaufman and Alm editors, published by Mosby, 2003, thatthe radius of curvature at the apex of the cornea is approximately 7.8mm and the eccentricity of the ellipsoid shape of the cornea isapproximately 0.5. The focal length of the corneal reflective surface atthe apex is therefore half this focal length: approximately 3.9 mm.Using the ellipsoidal model, the radius of curvature of the cornea at aradial distance of 6 mm from the apex of the cornea can be computed tobe approximately 9.6 mm. The focal length of the corneal reflectivesurface in this region is therefore approximately 4.8 mm. If the corneais situated approximately 150 mm from the computer screen, then from thestandard lens equation above, d1 can be computed to be 4.0 mm at theapex, and 4.96 mm at a radial distance of 6 mm from the apex of thecornea. The magnification is computed to be d1/d0=4.0/150=1/37.46 at theapex of the cornea, and 4.96/150=1/30.25 at a radial distance of 6 mmfrom the apex of the cornea. This means that the cornea has the effectof reducing the size of the graphic on the computer screen by a factorof 37.46 to 30.25 in this case, over different regions of the cornea,whereas the magnification expected if the reflective surface is flatis 1. If the detected graphic is significantly larger or smaller thanthe reduction factors 37.46 to 30.25, then the curvature of the corneais inconsistent with that of a live person.

If the local radius of curvature of the cornea is substantially lessthen the distance of the cornea to the computer screen, then themagnification can be simplified to be R/(2×d1), where d1 is the distancefrom the cornea to the computer screen and R is the local radius ofcurvature of the cornea. Due to human variation, the radius of curvatureof local regions of a cornea may lie within the bounds of 6 to 12 mm.The magnification therefore may lie in the range of 3/d1 to 6/d1.

In another example, if d1 lies within the range of 75 to 500 mm thenusing the parameters and the formula above, it is expected that themagnification is 1/12.5 to 1/166.7.

The distance d1 may be unknown, however, the ratio of the magnificationat the apex of the cornea and the magnification elsewhere in the corneais independent of the distance d1. For example, using the parametersabove, the magnification ratio between the apex and a point 6 mmradially from the apex is (1/37.46)/(1/30.25)=0.807. At a distance of 4mm radially from the apex, the expected magnification ratio is computedto be 0.909. The iris is approximately 11 mm in diameter, and thereforelocalization of the iris/sclera boundary can be used to identify theapproximate location of any radial position of the cornea with respectto the apex.

In another example, consider the change in configuration caused by themovement of a person with respect to a camera and one or moreilluminators or computer screens. The position of the reflection of theilluminators, or the detected shape and magnification of the computerscreen, will change as the person moves. In one preferred implementationto detect this change, a sequence of images are acquired and imagealignment is performed using a hierarchical, iterative method such asdescribed by Bergen et al., “Hierarchical Model-BasedMotion-Estimation,” European Conference on Computer Vision, 1993, whichis incorporated herein by reference in its entirety. For example, atranslation and zoom model can be applied between the warped images W(Q)and the original images O(R). In this case the motion of the user willbe stabilized, and, for example, the image of the iris may be alignedthroughout the sequence. Any residual motion is an indication of achange in the position of the reflection of the illuminators, or of achange in the shape and magnification of the computer screen, due to aneye consistent with that of a live person. For example, one preferredmethod of detecting the residual motion or change is shown in R. Kumar,P. Anandan, and K J. Hanna, “Direct Recovery of Shape from MultipleViews: a Parallax Based Approach,” Proceedings of the 12th IAPRInternational Conference on Pattern Recognition, vol. 1, pp. 685-688,1994, which is incorporated by reference in its entirety. In analternate method, a nonparametric flow model as described in Bergen etal., “Hierarchical Model-Based Motion Estimation,” European Conferenceon Computer Vision, 1993, can be applied to detect residual motion.

In another example, consider the presentation of illumination of aparticular wavelength, the recording of an image, and then presentationof illumination with a different wavelength and the recording of one ormore additional images. Depending on the photometric properties of thematerial, the ratio of the digitized intensities between the images canbe computed and compared to an expected ratio that has been previouslydocumented for that material. The response of iris tissue has a uniquephotometric signature which can indicate whether the iris is live ornot. Equally, the response of skin tissue has a unique photometricsignature which can indicate whether the skin is live or not. Thismethod can be implemented by acquiring two or more images with thecomputer screen projecting different wavelengths of light, such as red,green, and blue. These colors can be projected in a checkerboard orother pattern. For example, a first image may contain a red checkerboardpattern, and a second image may contain a blue checkerboard pattern. Themethods described above can then be used to align the images together,and to detect the location of the eye or eyes by detecting the patternsreflected off the cornea. The iris and the sclera (the white area aroundthe iris) are then detected. Many methods are known for detecting theiris and the sclera. For example, a Hough transform can be used todetect the circular contours of the pupil/iris and iris/scleraboundaries as explained by R. Wildes, “Iris Recognition: An EmergingBiometric Technology,” Proc IEEE, 85 (9): 1348-1363, September 1997,which is incorporated herein by reference in its entirety. Intensitiesof the iris and sclera can then be sampled and used to measure theliveness of the eye. These ratios can be computed in several ways. Inone preferred method, the ratio of the iris reflectance and the scleralreflectance is computed. This ratio is substantially independent of thebrightness of the original illumination. The iris/scleral ratio is thencomputed on the other aligned images. This process can be repeated bymeasuring the scleral/skin ratio. The skin region can be detected bymeasuring intensities directly under the detect eye position, forexample. Ratios can also be computed directly between correspondingaligned image regions captured under different illumination wavelengths.These ratios are then compared to pre-stored ratios that have beenmeasured on a range of individuals. One method of comparison is tonormalize the set of ratios such that sum of the magnitudes of theratios is unity. The difference between each normalized ratio and thepre-stored value is then computed. If one or more of the normalizedratios is different from the pre-stored ratio by more than a pre-definedthreshold ratio, then the measured intensity values are inconsistentwith those of a real eye.

In yet another example, the user may be asked to fixate on two or moredifferent locations on the computer screen while a single camera recordstwo or more images. The specular reflection off the cornea will remainsubstantially in the same place since the cornea is substantiallycircular, but the iris will appear to move from side to side in theimagery. In order to detect this phenomenon, the alignment methodsdescribed above can be used to align the images acquired when the useris looking in the first and second directions. The high-frequencyfiltering methods and the image differencing method described above canthen be used to identify the eye regions. The alignment process can berepeated solely in the eye regions in order to align the iris imagery.The residual misalignment of the specular image can then be detectedusing the methods described earlier.

Method Comparing One Configuration

Introduction:

In the previous section, images were captured using at least twodifferent geometrical or photometric configurations of components. Thecaptured image data (or features derived from the data) acquired usingeach configuration were compared to each other and a set of changeparameters between the captured image data were computed. The set ofchange parameters were then compared with those change parameters thatwere predicted using knowledge of the expected change in geometrical orphotometric configuration of the components. In a second method, imageryis captured using one geometric or photometric configuration ofcomponents. The captured image data (or features derived from the data)is compared with data that is predicted using absolute knowledge of theexpected geometrical or photometric configuration of the components.Both the first and second methods can optionally be combined.

To illustrate an example of the second method, consider that the shapeof the cornea results in a particular reflection onto the camera. Forexample, an image projected on the screen of computer L may berectangular, but if the candidate corneal surface is convex then theimage captured by imager C comprises a particular non-rectangular shape,that can be predicted from, for example, the ellipsoidal model describedearlier in this specification. This particular reflected shape can bemeasured using methods described below, and can be used to determinewhether the cornea has a particular shape or not. FIG. 3 shows the stepsof the second method. The second method can be implemented in severalways, and the components for one approach were described above. Thatapproach comprises the first step of projecting a random graphic patternon a computer screen, which may optionally be pre-deformed such that thereflected image off the cornea is substantially free of deformation. Thesecond step is then to perform pattern recognition on the image todetect the reflected graphic pattern. The expected radius of curvatureof the cornea is used to compute the expected deformation and expectedmagnification as described above. An eye is determined to be live if therandom pattern is detected at approximately the correct size and withthe expected deformation. A method for performing the detection of thepattern was described above.

To illustrate the combination of the first and second methods, considerthe previous example but also consider that the projected image Lchanges over time. Both the absolute comparison of the reflected imagewith the expected absolute reflection as well as the change over time inthe reflected image compared to the expected change over time can beperformed to validate the geometrical relationship and/or photometricrelationship between or within the corneal and iris or retinal surfaces.

Optimizing Performance:

As set forth above, the number and configuration of the various systemcomponents that include (I, L, C, A, B, (X, Y, T)) can vary widely, andthe methods are still capable of determining the parameters of thegeometrical and/or photometric relationship between or within eithersurface A, B which are the corneal and iris or retinal surfaces. Inorder to optimize the particular configuration of the various systemcomponents, many factors in the optimization need to be included, forexample: cost, size, and acquisition time. Depending on these variousfactors, an optimal solution can be determined. For example, consider anapplication where only one camera C can be used, the candidate cornealsurface A and iris or retinal surface B is fixed, and only a projectedlight source from a computer L can be used. Using the first method,variation in the configuration may be derived from the remainingconfiguration parameters (X, Y, T) and L. For example, the surfaces maymove in space and time, and imagery captured. In another example wherethe orientation and position of the eye (X, Y) is fixed, then theprojected image L can be varied, and imagery acquired by camera C. Notethat variation in all parameters can be performed simultaneously and notjust independently. All variations provide supporting evidence about theparametric relationship between or within the corneal and iris/retinalsurfaces that is used to determine whether an iris is live or not. Ifanyone of the measured variations does not equal the expected variation,then the iris is not live.

As mentioned above, it is well known that imagery of the iris can bereliably matched to previously recorded iris imagery in order to performreliable verification or recognition. However since the iris patternsare not easily recognizable to a human, it is impossible to demonstrateto a user who has been rejected from any iris recognition system thereason for the rejection. On the other hand, if a face recognitionsystem is used instead of an iris recognition system, it is easy todemonstrate the reason for rejection since face imagery can be easilyinterpreted by humans. However, automated face recognition systems arewidely known to be much less reliable than iris recognition systems.

We propose a method whereby iris imagery is acquired and used forautomatic iris matching, face imagery is acquired at least for thepurposes of human inspection generally in the case of rejection, andwhere the face and iris imagery is acquired and processed such that itis known that the face and iris imagery were derived from the sameperson. We present a design methodology and identify particular systemconfigurations, including a low-resolution single camera configurationcapable of acquiring and processing both face and iris imagery so thatone can confirm and corroborate the other, as well as give assurances tothe user who cannot properly interpret an iris image. We first present asingle-sensor approach.

Single-Sensor Approach:

Most methods for acquiring images of the face or iris use cameraimagers. The simplest method for acquiring imagery of the face and iriswith some evidence that the imagery is derived from the same person isto capture a single image of the face and iris from a single imager.However, in order to capture the face in the field of view, the numberof pixels devoted to the iris will be quite small, and not typicallysufficient for iris recognition. High resolution imagers can be used,but they are expensive and not as widely available as low-cost consumercameras. Also, the albedo (reflectance) of the iris is typically verylow compared to the face, and this means that the contrast of the irisis typically very small in cases where the full face is properly imagedwithin the dynamic range of the camera. It is generally much moredifficult to perform reliable iris recognition using a low-contrastimage of the iris. It is also more difficult to implement reliableanti-spoofing measures for iris recognition when the acquired data islow resolution or low contrast.

We propose a method whereby multiple images of the face and iris arecollected, the images are processed, and a determination is made as towhether the face and iris images are part of an expected sequence ofimages. If a determination can be made that the images were collected aspart of an expected sequence of images, then a determination can be madethat the face and iris imagery are of the same person. The multipleimages may be collected under different imaging conditions, for example:change in the midpoint of the camera's dynamic range; change in positionof the user; and/or change in zoom of the camera.

For example, FIG. 4 illustrates the second and third scenarios. A user“A” may present themselves to a simple consumer web-camera “C”. As theuser moves towards or away from the web-cam, images of the face and irisare acquired. S0, S10, S20, S30 are example images from the acquiredsequence. When the user is close to the web-cam, then the iris is imagedoptimally. The resolution and contrast of the iris will be substantialwhich is helpful to optimize recognition, and also makes anti-spoofingmeasures easier to implement. When the user is far from the web-cam, afull face image will be acquired.

Associating Face and Iris Imagery:

We now describe a method for associating face and iris imagery in asequence. FIG. 5 shows the approach. The first step is to have knowledgeof the expected imaging scenario “I”. For example, in the example above,the expected imaging scenario is that the user will approach or moveaway from the camera (as shown, for example, in FIG. 4). Next, the imagesequence is acquired as shown in step A. Next, image processing isperformed between the images in the sequence to produce parameters asshown in step M. For example, continuing with the example above, motioncan be computed between frames and a similarity (zoom, translation,rotation) model of motion can be fit. The expected set of parameters isderived in step E using knowledge of the imaging scenario. For example,we would expect to see a substantial zoom component in the motionanalysis parameters as the user approaches or moves away from thecamera, and we would expect there to be a continuous single motion andnot two motions. For example, if the user tried to insert a picture of asecond person into the camera's field of view as they moved away fromthe camera, then two motions would be present—one due to the user movingback, and the second from the insertion of the picture into the view.The measured and expected parameters are then compared in step D. Ifthere is a significant difference between the expected and measuredparameters, then the face and iris imagery cannot be associated. Ifthere is not a significant difference, then the face and iris imagerycan be associated.

It is important that the method track and perform alignment from theimage at or close to the iris image used for biometric recognition toanother image taken later or earlier in the sequence of images. Ideallybut not necessarily, image tracking would be from the actual image usedfor iris matching. However, using the very same image used for irismatching is not required. The key constraint is that the iris image ator near the matched iris image has to be close enough in time to preventa user from suddenly switching the camera from one person to the next,or from inserting a picture of someone else's iris or face, withoutdetection. If the frame rate were as low as 0.5 Hz, then it is plausibleto see that this could happen. A preferred time interval between framesis thus 2 seconds or less.

The image acquiring process of the method must include acquiring an irisimage suitable for biometric recognition. The determination of whatconstitutes a suitable image can be made automatically by known methods.Once that suitable iris image has been determined to have been acquired,as mentioned above, tracking and alignment must be performed between atleast that iris image (or a nearby image in the sequence) and anotherimage, e.g., the image at the other end of the sequence where the irisimage is at one end. The other image is described as being preferably animage of the user's face, however it need not be so limited. The otherimage used for tracking an alignment could be an image of the whole bodyof a person, a place, a face, an ID number on a wall, or pretty muchanything that allows for confirmation of the user's iris image in amanner that is perceptible to the human eye. The selection of that otherimage can be accomplished in one or more of several ways. For example,it could be selected manually (e.g., by a button press holding thedevice far away or at a target), and then the end of the sequence (wherethe iris imagery is acquired) is detected automatically. As anotherexample, it could also be selected via an automatic face findingalgorithm, or an automatic symbol detection algorithm. The selection canalso be made using the zoom difference between a face image and an irisimage, since if an iris image is selected, then taking an image at least10 times zoomed out and in the same location will result in the face.

Regarding this last method, if an iris image is selected, one can besure there is a face in the image without doing processor-intensive facefinding if the zoom and position parameters of images in the sequenceare examined. If the position hasn't moved by more than the field ofview of the camera, and the zoom is a certain amount, then the face issurely in the field of view. Put another way, if there are N pixelsacross the iris when matched (the ISO standard for N is in the range of100 to 200 pixels), and P pixels between the eyes in the face image aredesired (also 100-200 pixels, per ISO standards), then we wait until thezoom difference measured is approximately 10, since the ratio of thetypical iris to the typical eye separation is about 10.

Example Implementation Methods:

There are many methods for performing steps M, E, C, D in FIG. 5. Wepresent a preferred set of methods as an example. First, if the imagesacquired in step A are labeled 11, 12, 13, 14 etc., then a differenceimage sequence D1, D2, D3, D4 is computed by subtracting adjacentframes. For example, D1=12-11, and D2=13-12 etc. If the camera sensor isstationary, then this differencing removes the background fromprocessing meaning that all resultant intensities in the sequence aredue to the moving person. Note that this approach is only used if onlythe user is moving. In cases of a lens that zooms, this step would notbe performed since the background also moves. Second, flow analysis isperformed between successive image pairs. There are many methods knownin the art for performing flow analysis. An example is Bergen et. al,“Hierarchical Motion Analysis”, European Conference on Computer Vision,1992. We compute both flow values and also confidence measures in theflow at each location in the image for each image pair consisting of areference and an inspection image. Third, we fit a similarity transform(translation, zoom and rotation) model to the recovered flow vectors,accounting for the confidence values using a lest-squares model-fitmethod. For example in the case of a moving user, regions due to thebackground have zero confidence since there are no intensities due tothe use of the first image differencing step. A RANSAC fitting algorithmthat is robust to outliers can be used [M. A. Fischler, R. C. Bolles.Random Sample Consensus: A Paradigm for Model Fitting with Applicationsto Image Analysis and Automated Cartography. Comm. of the ACM, Vol 24,pp 381-395, 1981.]. Fourth, once the model has been fit, we warp theinspection image to the reference image using the model and compute theresidual misalignment or difference between the warped image and thereference image. For example, we can re-compute the flow analysisbetween the warped image and the reference image. Any motion that isinconsistent with a person moving towards or away from the camera (suchas the motion from a person's picture being inserted into the sequence)will be measured. We then histogram the magnitudes of these residualintensities, and repeat the entire process for every successive imagepair in the sequence. For additional sensitivity, we also repeat themotion analysis between non-adjacent image pairs using the computedcascaded motion parameters between frames as a seed to begin the motionanalysis. Finally, we inspect the histogram of the residual motions ordifferences. If there are any residual motions or differences above athreshold value, then we declare that the face and iris imagery cannotbe associated.

In addition, we can use the recovered model parameters to ensure that aface image has actually been acquired. For example, an iris finderalgorithm may have located the iris precisely. The motion parametersfrom the model-fitting process defined above can be cascaded in order topredict whether a full face image is in fact visible in any part of thesequence by predicting the coverage of the camera on the person's facefor every image with respect to the location of the iris. For example,we may measure a translation T and a zoom Z between an image containingthe iris at location L and a second image.

We can then predict the face coverage on the second image using theparameters T, Z and L and the typical size of a person's head comparedto their iris. For example, an iris is typically 1 cm in diameter and aperson's head is typically 10 cm in diameter. A zoom factor ofapproximately 10 between the iris image and the second image willindicate that the second image is at least at the correct scale tocapture an image of the face. The translation parameters can beinspected similarly. This inspection method can also be used to stop theacquisition process given an initial detection of the iris.

Multi-Sensor Approach:

The single-sensor approach above can be extended to the use of multiplesensors. For example, FIG. 6 shows two sensors focused on a person.

One imager may have higher resolution than the other. We now can performimage processing both within a single sequence and also between the twosequences. For example, if one imager is low resolution and the secondimager is high resolution, then the parameters of motion recovered fromeach image sequence using the methods described above will be directlyrelated—for example if the imagery moves to the right in thelow-resolution imager, then the imagery will move to the right at afaster speed in the second imager. If this does not occur, then theimagery being sent from each imaging device are likely not derived fromthe same person. This is important in non-supervised scenarios wherevideo connections to the two sensors may be tampered with. In additionto comparison of motion parameters between sequences, images themselvescan be compared between sequences. For example, if the approximate zoomfactor between the high and low resolution cameras is known, then thehigh resolution image can be warped to the resolution of the lowresolution image, and an image correlation can be performed to verifythat the imagery from the two or more sensors are in fact derived fromthe same scene.

In the foregoing specification, embodiments of the invention have beendescribed with reference to numerous specific details that may vary fromimplementation to implementation. Thus, the sole and exclusive indicatorof what is the invention, and is intended by the applicants to be theinvention, is the set of claims that issue from this application, in thespecific form in which such claims issue, including any subsequentcorrection. Any definitions expressly set forth herein for termscontained in such claims shall govern the meaning of such terms as usedin the claims. Hence, no limitation, element, property, feature,advantage or attribute that is not expressly recited in a claim shouldlimit the scope of such claim in any way. The summary, specification anddrawings are, accordingly, to be regarded in an illustrative rather thana restrictive sense.

What is claimed is:
 1. A method of biometric acquisition, comprising:acquiring, by a first sensor, an image of a face; acquiring, by a secondsensor, an image of a corresponding iris suitable for computerrecognition at the same time as or within a specified period of timefrom acquiring the image of the face to ensure that the image of theface and the image of the corresponding iris are from a same liveperson; associating the image of the face with the image of thecorresponding iris for biometric identification of the same live person;and determining if motion vectors from comparing the image of thecorresponding iris with the image of the face are consistent with anexpected sequence of images acquired from the same live person.
 2. Themethod of claim 1, wherein one of the first sensor and the second sensoris associated with a resolution higher than that of another of the firstsensor and the second sensor.
 3. The method of claim 1, furthercomprising performing image correlation between portions of the image ofthe face and the image of the corresponding iris.
 4. The method of claim3, comprising verifying that the image of the face and the image of thecorresponding iris are derived from a same scene or person, according tothe image correlation.
 5. The method of claim 1, further comprisingdetermining if at least a portion of the iris is present in both theimage of the corresponding iris and the image of the face.
 6. The methodof claim 1, wherein acquiring the image of the face comprises acquiringthe image of the face suitable for human recognition.
 7. The method ofclaim 1, further comprising having a human inspector inspect the imageof the face responsive to computer recognition of the image of thecorresponding iris, the inspection and recognition of the image of theface and the image of the corresponding iris performed against storedimages corresponding to the same live person.
 8. The method of claim 1,further comprising determining if the image of the corresponding irisincludes a reflection that is consistent with one from a human eye. 9.The method of claim 1, further comprising determining a resolution orzoom difference between the image of the face and the image of thecorresponding iris.
 10. The method of claim 1, comprising acquiring theimage of the corresponding iris to have a resolution of 100 to 200pixels across the corresponding iris.
 11. A system, comprising: a firstsensor configured for acquiring an image of a face; a second sensorconfigured for acquiring an image of a corresponding iris suitable forcomputer recognition at the same time as or within a specified period oftime from acquiring the image of the face to ensure that the image ofthe face and the image of the corresponding iris are from a same liveperson; and a processor, operably coupled to the first sensor and thesecond sensor, configured to associate the image of the face with theimage of the corresponding iris for biometric identification of the samelive person, and to determine if motion vectors from comparing the imageof the corresponding iris with the image of the face are consistent withan expected sequence of images acquired from the same live person. 12.The system of claim 11, wherein one of the first sensor and the secondsensor is associated with a resolution higher than that of another ofthe first sensor and the second sensor.
 13. The system of claim 11,wherein the processor is configured to perform image correlation betweenportions of the image of the face and the image of the correspondingiris.
 14. The system of claim 13, wherein the processor is configured toverify that the image of the face and the image of the correspondingiris are derived from a same scene or person, according to the imagecorrelation.
 15. The system of claim 11, wherein the processor isconfigured to determine if at least a portion of the iris is present inboth the image of the corresponding iris and the image of the face. 16.The system of claim 11, wherein the first sensor is configured toacquire the image of the face, the acquired image of the face beingsuitable for human recognition.
 17. The system of claim 11, wherein theimage of the face is inspected responsive to computer recognition of theimage of the corresponding iris, the inspection and recognition of theimage of the face and the image of the corresponding iris performedagainst stored images corresponding to the same live person.
 18. Thesystem of claim 11, wherein the processor is configured to determine ifthe image of the corresponding iris includes a reflection that isconsistent with a reflection from a human eye.
 19. The system of claim11, wherein the processor is configured to determine a resolution orzoom difference between the image of the face and the image of thecorresponding iris.
 20. The system of claim 11, wherein the secondsensor is configured to acquire the image of the corresponding iris, theacquired image of the corresponding iris having a resolution of 100 to200 pixels across the corresponding iris.